Medical image processing apparatus and medical image processing method

ABSTRACT

A medical image processing apparatus according to an embodiment includes acquisition circuitry, extraction circuitry, calculation circuitry, and determination circuitry. The acquisition circuitry configured to acquire a first photographic image and a second photographic image that contain tree structures of a subject. The extraction circuitry configured to extract branch points of each of the tree structures in the first photographic image and the second photographic image. The calculation circuitry configured to calculate the similarities between the branch points in the first photographic image and the branch points in the second photographic image based on the feature quantities of the branch points. The determination circuitry configured to determine the corresponding path between the branch points in the first photographic image and the branch points in the second photographic image based on the similarities between the branch points calculated by the calculation circuitry.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromChinese Patent Application No. 201410602444.2, filed on Oct. 31, 2014and Japanese Patent Application No. 2015-171078, filed on Aug. 31, 2015;the entire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate to a medical image processingapparatus and a medical image processing method.

BACKGROUND

In recent years, there have been medical image processing apparatusesthat resolve the conditions of chronic obstructive pulmonary disease(COPD), emphysema, tracheal and bronchial diseases and the like. Forexample, a medical image processing apparatus matches the positions(matching) of a plurality of images containing tree structures andacquired in different phases of the bronchial three-dimensional (3D)images. However, there are some physiological and pathologicaldifferences among 3D images, and variations are present between 3Dimages in different phases due to motions such as respiration. Thisleads to difficulty in realizing automatic position matching between 3Dimages and problems, such as sensitivity to noise, a large amount ofcomputation and so on.

According to Patent Literature 1 (U.S. Pat. No. 7,646,903), tree-likestructures representative of physical objects or models are acquired, apath is extracted from a tree-like structure, a path is extracted fromanother tree-like structure, the two paths are compared to each other bycalculating the similarity measurement results, and it is determinedwhether the paths match up based on the similarity measurement results.

According to Patent Literature 2 (U.S. Laid-open Patent Publication No.2012/0263364), the matching algorithm is based on association diagrammethod, and the computation time is significantly reduced by introducinghierarchic separation as well as matching only two sub-trees at once.

However, the disadvantage of the technology described in PatentLiterature 1 is in sensitivity to noise and incorrect central line andin that there are too many feature points and too much computation load.

The disadvantage of the technology described in Patent Literature 2 isin that the primary branch points are sensitive to noise and incorrectcentral line and solving the association diagram is a difficulty in NPsolution and there is too much computation load.

Exemplary position matching results according to the prior art will bedescribed. FIG. 12 is a diagram showing exemplary position matchingresults according to the prior art. When position matching using animage acquired in a phase (hereinafter, referred to as Phase 1) as abase image is performed, the actual position matching results in animage acquired in another phase (hereinafter, Phase 2) for the path ofthe user's interest marked with “2” (the left diagram in FIG. 12) in theimage acquired in Phase 1 are the path marked with “2” (the centerdiagram in FIG. 12). The position matching results desired for Phase 2are, as shown in the right diagram in FIG. 12, the path marked with “3”.As described above, it can be seen that the path marked with “2” that isthe actual matching results in the image acquired in Phase 2significantly shifts to the left.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an exemplary configuration of amedical image processing apparatus according to a first embodiment;

FIG. 2 is a flow chart showing an exemplary medical image processingmethod performed by the medical image processing apparatus according tothe first embodiment;

FIG. 3 is a schematic diagram for explaining exemplary position matchingof branch points of tree structures according to the embodiment;

FIG. 4 is a diagram for explaining an exemplary modification of thefirst embodiment;

FIG. 5 is a flow chart showing an exemplary position matching processaccording to a second embodiment;

FIG. 6 is a schematic diagram showing exemplary extracted featurepoints;

FIG. 7 is a schematic diagram showing exemplary position matchingresults;

FIG. 8 is a block diagram showing an exemplary configuration of anacquisition unit according to a third embodiment;

FIG. 9 is a flow chart showing an exemplary flow of acquiring treestructures in various levels according to the third embodiment.

FIG. 10 is a block diagram showing an exemplary configuration of amedical image processing apparatus according to a fourth embodiment;

FIG. 11 is a diagram showing exemplary position matching performed in acertain area by the medical image processing apparatus; and

FIG. 12 is a diagram showing exemplary position matching results in theprior art.

DETAILED DESCRIPTION

A medical image processing apparatus according to an embodiment includesacquisition circuitry, extraction circuitry, calculation circuitry, anddetermination circuitry. The acquisition circuitry configured to acquirea first photographic image and a second photographic image that containtree structures of a subject. The extraction circuitry configured toextract branch points of each of the tree structures in the firstphotographic image and the second photographic image. The calculationcircuitry configured to calculate the similarities between the branchpoints in the first photographic image and the branch points in thesecond photographic image based on the feature quantities of the branchpoints. The determination circuitry configured to determine thecorresponding path between the branch points in the first photographicimage and the branch points in the second photographic image based onthe similarities between the branch points calculated by the calculationcircuitry.

The determination circuitry of the medical image processing apparatusaccording to the embodiment is configured to determine the correspondingbranch points between the first photographic image and the secondphotographic image based on the similarities between the branch pointscalculated by the calculation circuitry and take the path consisting ofthe respective determined branch points as the corresponding pathbetween the tree structures.

When the determination circuitry is configured to determine a pluralityof paths, the calculation circuitry of the medical image processingapparatus according to the embodiment is configured to calculate thesimilarities between a reference path in the first photographic imageand each of the paths in the second photographic image, based on a setof the feature quantities of various branch points constituting thepaths, and the determination circuitry is configured to determine thecorresponding path between the tree structures, based on thesimilarities between the various paths calculated by the calculationcircuitry.

The acquisition circuitry of the medical image processing apparatusaccording to the embodiment is configured to include: multi-levelacquisition circuitry configured to divide a tree structure into aplurality of levels toward outside from the root of the tree structure,taking the level where the seed points locate as start points, acquirethe corresponding portions of the tree structure in the respectivelevels in turn, and combine the acquired portions into a complete treestructure.

The method determination circuitry of the medical image processingapparatus according to the embodiment is configured to evaluate theacquisition results on the former level and, based on the result of theevaluation, determine the acquisition method to be performed by themulti-level acquisition circuitry for the current level.

The method determination circuitry of the medical image processingapparatus according to the embodiment is configured to take theacquisition method for the level where the seed points locate as regiongrowing and repeat a process of adaptively acquiring the segmentationresults of the current level by adaptively adjusting the parameters ofthe current level based on the feedback of the segmentation results onthe former level until the growth of the whole tree structure has beencompleted.

The medical image processing apparatus according to the embodimentfurther includes division circuitry configured to divide the treestructure into a plurality of areas, wherein the extraction circuitry isconfigured to extract the branch points in the areas based on the areasdivided by the division circuitry.

The division circuit of the medical image processing apparatus accordingto the embodiment is configured to determine an area of concern, and theextraction circuitry configured to extract the branch points for onlythe area of concern.

The determination circuit of the medical image processing apparatusaccording to the embodiment is configured to take the branch pointsextracted by the extraction circuitry as branch point candidates andrank the various branch point candidates based on the similaritiesbetween the branch points.

The calculation circuitry of the medical image processing apparatusaccording to the embodiment is configured to calculate the similaritiesbetween the various branch points based on the positions of the branchpoints in the tree structure after weighting the various branch points.

The calculation circuitry of the medical image processing apparatusaccording to the embodiment is configured to divide the tree structureinto levels based on the growth levels of the tree structure andcalculate only the similarities between the branch points in the samelevel.

Furthermore, a medical image processing apparatus according to anembodiment includes multi-level acquisition circuitry and methoddetermination circuitry. The multi-level acquisition circuitry isconfigured to divide a tree structure into a plurality of levels, takingthe level where the seed points locate as start points, acquire thecorresponding portions of the tree structure in the respective levels inturn, and combine the acquired portions into a complete tree structure.The method determination circuitry is configured to evaluate theacquisition results on the former level and, based on the results of theevaluation, determine the acquisition method to be performed by themulti-level acquisition circuitry for the current level.

A medical image processing method according to an embodiment includes:by acquisition circuitry, acquiring a first photographic image and asecond photographic image that contain tree structures of a subject; byextraction circuitry, extracting branch points of each of the treestructures in the first photographic image and the second photographicimage; by calculation circuitry, calculating the similarities betweenthe branch points in the first photographic image and the branch pointsin the second photographic image based on the feature quantities of thebranch points; and, by determination circuitry, determining thecorresponding path between the branch points in the first photographicimage and the branch points in the second photographic image based onthe similarities between the branch points calculated by the calculationcircuitry.

A medical image processing method according to an embodiment includes:by multi-level acquisition circuitry, dividing a tree structure into aplurality of levels, taking the level where the seed points locate asstart points, acquiring the corresponding portions of the tree structurein the respective levels in turn, and combining the acquired portionsinto a complete tree structure; and, by method determination circuitry,evaluating the acquisition results on the former level and, based on theresults of the evaluation, determining the acquisition method to beperformed by the multi-level acquisition circuitry for the currentlevel.

The embodiments will be described in detail below with reference to thedrawings. The following descriptions will take tree structures ofbronchus as an example. However, the embodiments are not limited to theprocessing on the images representing the tree structures of bronchus,i.e., including but not limited to the processing for trachea andbronchus, and it can also include the processing on the imagesrepresenting other tree structures such as the structures of cranialnerve and blood vessel.

First Embodiment

First, a medical image processing apparatus 1 of an embodiment will bedescribed. FIG. 1 is a block diagram showing an exemplary configurationof a medical image processing apparatus according to the firstembodiment. As shown in the example in FIG. 1, the medical imageprocessing apparatus 1 includes an acquisition unit 11, an extractionunit 12, a calculation unit 13, and a determination unit 14. Theacquisition unit 11, the extraction unit 12, the calculation unit 13,and the determination unit 14 are realized by a processor. The processorincludes circuitry.

The acquisition unit 11 acquires a first photographic image and a secondphotographic image containing tree structures of a subject in differentphases. Here, the acquisition unit 11 extracts the tree structures fromthe photographic images with various existing methods of extracting atree structure.

The extraction unit 12 extracts the branch points of the various treestructures contained in the first photographic image and in the secondphotographic image. A branch point is a point representing the rootwhere a branch locates in a tree structure.

Furthermore, in the present embodiment, in the case of displaying treestructures with central lines of tree stem, the convergence points ofdifferent central lines are taken as branch points. The calculation unit13 calculates the similarities between the branch points in the firstphotographic image and the branch points in the second photographicimage based on the feature quantities of the branch points.

The calculation unit 13 is capable of calculating the feature quantitiesof the various branch points in tree structures by comparison. Forexample, in the case that the region of the subject represented by thetree structure is bronchus, the calculation unit 13 selects certainpreferable feature quantities, such as coordinates, direction, andbranch level, from among a plurality of feature quantities of the branchpoints based on the calculation ability of the calculation unit 13 etc.,and calculates the similarities by using a comparison function.

The determination unit 14 determines the corresponding path between thetree structure in the first photographic image and the tree structure inthe second photographic image based on the similarities between thebranch points calculated by the calculation unit 13.

The medical image processing apparatus 1 may be realized with a computerby reading a program and executing the read program to carry out thefunctions of the above-described various components, and may also berealized by the hardware configurations, such as integrated circuits,forming the above-described various components respectively.

The medical image processing method performed by the medical imageprocessing apparatus 1 according to the present embodiment will bedescribed below. FIG. 2 is a flow chart showing the exemplary medicalimage processing method performed by the medical image processingapparatus 1 according to the first embodiment.

As shown in FIG. 2, the medical image processing method performed by themedical image processing apparatus 1 includes an acquisition step S1, anextraction step S2, a calculation step S3 and a determination step S4.In the acquisition step S1, the acquisition unit 11 acquires a firstphotographic image and a second photographic image that contain the treestructures of a subject. In the extraction step S2, the extraction unit12 extracts the branch points of each of the tree structures containedin the first photographic image and in the second photographic image. Inthe calculation step S3, the calculation unit 13 calculates thesimilarities between the branch points in the first photographic imageand the branch points in the second photographic image based on thefeature quantities of the branch points. In the determination step S4,the determination unit 14 determines the corresponding path between thebranch points in the first photographic image and the branch points inthe second photographic image based on the similarities between thebranch points calculated in step S3.

One specific example of the medical image processing method performed bythe medical image processing apparatus 1 according to the firstembodiment will be described below. The specific example gives anexample just to make it easier to understand the embodiment and is notintended to limit the embodiment.

In the specific example, in the acquisition step S1, the acquisitionunit 11 acquires the central line trees in the photographic images inall phases. The central line tree so-called herein is referred to as thetree structure representing the tree stem as the central line.

Then, in the extraction step S2, the extraction unit 12 extracts thebranch points of the portion of the central line in the photographicimage in a first phase (hereinafter, it may be referred to as a firstphotographic image) and the branch points of the portion of the centralline in the photographic image in a second phase (hereinafter, it may bereferred to as a second photographic image). The first phase is thephase in which a photographic image containing the path serving as areference for performing position matching is acquired. The path servingas the reference for performing position matching is also referred to asa reference path. The reference path is the path of the user's interest.The second phase is a phase in which a photographic image containing apath for which position matching with the reference path is to beperformed is acquired.

The calculation unit 13 performs the calculation step S3. The exemplarycalculation step S3 of the embodiment will be described with referenceto FIG. 3. FIG. 3 is a schematic diagram for explaining the exemplaryposition matching of the branch points of the tree structures accordingto the embodiment. The process of determining the path corresponding tothe reference path by calculating the similarities between the branchpoints will be described with reference to FIG. 3. The position matchingobject in FIG. 3 is the path (fa0-fa1-fa21-fa31-fa42) in the firstphotographic image. In other words, in the example in FIG. 3, the objectis to obtain the path corresponding to the path (fa0-fa1-fa21-fa31-fa42)in the first photographic image from the second photographic image byposition matching.

In the case of marking the branch points with the features of the branchpoints, given the feature quantity of the branch points in the firstphotographic image is fa, and given the feature quantity of the branchpoints in the second photographic image is fb. First, the similaritiesbetween the branch points fa1 of the first photographic image in thefirst phase and the branch point fb1 of the second photographic image inthe second phase are calculated.

For the specific example of the feature quantity, for example, thefeature of each branch point is f_(n)=[x1,x2,x3,x4,x5,x6,x7]^(T), i.e.,is represented by an array of 7 features, wherein x1 represents thebranch point position, x2 represents the angle of the vertex formed bythe horizontal direction and any one of the central lines branched offfrom the branch point, x3 represents the length of fragment (e.g., thetotal of the length of two of the central lines branched off from thebranch point), x4 represents the angle of fragment (e.g., the angle ofthe vertex formed by two of the central lines branched off from thebranch point), x5 represents the similarities between a template imagerepresenting the cross section of bronchus and a cross-sectional imagecontaining the branch points and orthogonal to the central line, and x6represents the area of torus of the cross section containing the branchpoints and orthogonal to the central line, and x7 represents a profileparameter (e.g., circularity of torus in the cross section containingthe branch points and orthogonal to the central line).

The calculation for similarities is performed based on the informationof the tree structure branch points, such as the area of cross section.For example, in the case that given the feature quantity of a branchpoint in the first photographic image is fa1 and given the featurequantity of a branch point in the second photographic image is fb1, thenthe similarities between the two branch points are represented by thefunction S(fa1,fb1). Here, S(fa1,fb1)>δ, where δ is the preset thresholdand the threshold 5 is acquired in advance based on experience orstatistical results.

When the resulting similarities from the calculation is greater than δ,the search for the branch points in the subsequent level is proceededon, followed by step S2.

In step S2, the similarities between fa21 and fb21, fb22 is calculated,where the results are

S(fa21,fb21)>δ, and S(fa21,fb22)>δ.

Because S(fa21,fb21)>δ, S(fa21,fb22)>δ and the similarities are greaterthan the preset threshold, the selected branch points fb21, fb22 arevalid. If it is the case that similarities are less than the presetthreshold, it is terminated at the branch points i.e., the search forthe branch points in the subsequent level is ceased.

Proceed to step S3, where the similarities between fa31 and fb31, fb32,fb33, fb34 are calculated and the results are

S(fa31,fb31)>δ,

S(fa31,fb32)<δ,

S(fa31,fb33)<δ,

S(fa31,fb34)<δ.

Here, at the branch points b32, b33, b34 where the similarities are lessthan δ, further search for the branch points in the subsequent level isceased.

In step S4, the similarities between fa42 and fb41, fb42 are calculatedwhere the results are

S(fa42,fb41)<δ,

S(fa42,fb42)>δ.

Then, the determination unit 14 determines the corresponding branchpoints between the first photographic image and the second photographicimage based on the calculated similarities of the branch points in stepS4, and takes the path consisting of the various determined branchpoints as the corresponding path between the tree structures, i.e.,through the above-described calculation step S3, the path finallydetermined by the determination unit 14 is b0->b1->b21->b31->b42 in thedetermination step S4. In this manner, the position matching over thebronchus area (lung) is performed.

The medical image processing apparatus 1 determines a corresponding pathbetween a first photographic image and a second photographic image bythe above-described position matching. The medical image processingapparatus 1 determines corresponding paths in a plurality of phases byrepeatedly performing the position matching and analyzes the bronchirepresented by the determined paths over time, thereby outputting theanalysis results over time. For example, the medical image processingapparatus 1 generates a graph representing variations in the scale ofthe area of the cross section of bronchus in the determined paths anddisplays the generated graph. Alternatively, the medical imageprocessing apparatus 1 may generate a video image of the cross sectionof the bronchus in the determined paths and display the generated videoimage.

The medical image processing apparatus 1 and the medical imageprocessing method of the first embodiment enables fully automatic andhighly precise position matching between images containing treestructures and enables higher robustness to noise and incorrect centralline in position matching.

Further, the corresponding branch points between the first photographicimage and the second photographic image can be determined based on thesimilarities of the calculated branch points, and the path consisting ofthe determined various branch points can be determined precisely as thecorresponding path between the tree structures.

Further, according to the embodiment, since the path is determined afterthe similarities are calculated for only the branch points and thebranch points from and after the branch point with similarity smallerthan the threshold are removed continuously in the calculation process,so the computation load is substantially reduced compared to that in theprior art, so the efficiency of calculation can be improved.

Further, for sake of description, a description is made for only theposition matching between images in two different phases performed bythe medical image processing apparatus 1. However, the medical imageprocessing apparatus 1 may also acquire a plurality of images in threeor more phases and select one as the first photographic image in thefirst phase from the acquired images while taking the other images asthe second photographic images, respectively, to match each of thesecond photographic images with the first photographic image. Theposition matching may also be performed between a plurality ofphotographic images, thereby the position matching of a set ofphotographic images is completed.

(Modification of First Embodiment)

The calculation unit 13 may calculate only the similarities between thebranch points in the same level based on the levels of the treestructures. Such a modification will be described as a modification ofthe first embodiment. FIG. 4 is a diagram for explaining the exemplarymodification of the first embodiment.

As shown in the example in FIG. 4, for each of the levels from the topto the bottom of the tree structures, the calculation unit 13 finds out,as candidate branch points, branch points corresponding to variousbranch points (reference branch points) in the first photographic imageacquired in the first phase from the second photographic image acquiredin the second phase. The calculation unit 13 takes branch points withsimilarities to the reference branch points greater than a certainthreshold as candidate branch points, thereby finally determining theroute consisting of the candidate branch points.

Second Embodiment

According to the first embodiment and the modification of the firstembodiment, the determination unit 14 determines a path consisting ofbranch points with similarities greater than the threshold in the secondphotographic image. For this reason, in the case that a plurality ofpaths consist of the branch points with similarities greater than thethreshold in the second photographic image, it may be assumed that aplurality of paths are determined in the second photographic image. Anembodiment where, in the case that a plurality of path is determined,one of the paths is determined at last will be described as a secondembodiment. The second embodiment will be described with reference toFIGS. 5, 6, 7 and 8. The block diagram of the second embodiment is thesame as that of the first embodiment, but the operations of theextraction unit 12, the calculation unit 13 and the determination unit14 according to the second embodiment consist of the operations to bedescribed below in addition to the operations of the extraction unit 12,the calculation unit 13 and the determination unit 14 according to thefirst embodiment, which will be described in detail below. Furthermore,the contents of the descriptions of the first embodiment will not berepeated and the corresponding descriptions in the first embodiment willbe cited.

In the case that the determination unit 14 determines a plurality ofpaths in a second photographic image, the calculation unit 13 of theembodiment calculates the similarities between the path of the treestructure in the first photographic image and each of the paths of thetree structure in the second photographic image based on a set offeature quantities of various branch points constituting the path. Thedetermination unit 14 determines the corresponding path between the treestructure in the first photographic image and the tree structure in thesecond photographic image based on the similarities between the path ofthe tree structure in the first photographic image and each of the pathsof the tree structure in the second photographic image that arecalculated by the calculation unit 13.

One specific example of the medical image processing method performed bythe medical image processing apparatus 1 of the second embodiment willbe described below. The specific example is taken as an example just tomake it easier to understand the embodiment and is not intended to limitthe embodiment.

FIG. 5 is a flow chart showing the exemplary position matching processaccording to the second embodiment. FIG. 6 is a schematic diagramshowing extracted exemplary feature points. FIG. 7 is a schematicdiagram showing exemplary position matching results.

The position matching process shown in FIG. 5 is executed in the casethat the determination unit 14 determines a plurality of paths in asecond photographic image. As shown in FIG. 5, based on a set of featurequantities of various branch points constituting the path, thecalculation unit 13 calculates the similarities between the path of thetree structure in the first photographic image and each of the paths ofthe tree structure in the second photographic image (step S100). Forexample, when the central line represents the tree structure, a set offeature quantities of all the branch points constituting a certaincentral line represents the feature quantity of the path, thereby thesimilarities between the reference path in the photographic image in thefirst phase and each of the paths of the photographic image in thesecond phase are calculated. The paths in the photographic image in thesecond phase refer to a plurality of paths determined by thedetermination unit 14 in the second photographic image. Because thepaths are candidate paths for the reference path, they are also referredto as candidate paths.

The determination unit 14 ranks the path candidates in descending orderaccording to the similarities (step S200). The determination unit 14selects a candidate path with the highest similarity as the pathcorresponding to the reference path and takes the selected candidatepath as position matching results or selects a certain number ofcandidate paths from the top in descending order according to thesimilarities and takes the selected candidate paths as the positionmatching results (step S300).

Further, the determination unit 14 may weight the feature quantities ofthe various branch points constituting the path, thereby calculating thesimilarities more precisely. A specific example will be described withreference to FIG. 6 below.

As shown in the example in FIG. 6, the extraction unit 12 first extractsthe features of the paths. Given the features of various branch pointsare fn, then the feature vector of the path is a set of featurequantities of all the branch points, which is represented as: Vr=[f0,f1, f2, f3, f4], where “0, 1, 2, 3, 4” represents the orientation of thepaths, where 0 represents the first branch point starting from the root,1, 2, 3, 4 represents the subsequent branch point of the path startingfrom the former branch point in turn. In other words, “0, 1, 2, 3, 4”here correspond to different levels of the paths (the branch levels ofthe path).

As the importance of the branch points differ on different levels, thefeature points have to be weighted according to the levels: Vr=[w0*f0,w1*f1, w2*f2, w3*f3, w4*f4], where w0, w1, w2, w3, w4 represents theweights corresponding to the levels. For example, the weight may beincreased as level increases. Note that “*” represents an operator.

Next, the calculation unit 13 performs the position matching algorithmof the tree search based on the features of the branch points.

Here also given the features of the various branch points aref_(n)=[x1,x2,x3,x4,x5,x6,x7]^(T).

The calculation unit 13 obtains the feature vector of the reference path(hereinafter, referred to as a reference feature vector) with thefeature vectors of the various branch points constituting the referencepath. The calculation unit 13 further obtains the feature vector of eachcandidate path (hereinafter, referred to as a candidate feature vector)with the feature vectors of various branch points constituting each ofthe candidate paths. The feature vector Vn of each candidate path isrepresented as follows.

Vn=[w0′*f0,w1′*f1,w2′*f2,w3′*f3,w4′*f4],

where w0′, w1′, w2′, w3′ and w4′ are weights corresponding to thelevels.

Then, the calculation unit 13 may calculate, per candidate featurevector, the similarities between two vectors (the reference featurevector and the candidate feature vector) in the feature space based onthe reference feature vector and each of the candidate feature vectors.To calculate the similarities between two vectors, for example, standardsimilarity measurement and other similarity measurements may beemployed.

The determination unit 14 may rank the candidate paths based on thesimilarities per candidate feature vector. For example, thedetermination unit 14 may rank the candidate paths such that that rankincreases as the similarity increases. The determination unit 14 mayoutput the ranked candidate paths. Finally, the determination unit 14may select the candidate path with the highest similarities as the pathcorresponding to the reference path and take the selected candidate pathas the position matching results, or select a certain number ofcandidate paths in descending order according to the similarities andtake the selected candidate paths as the position matching results.

Accordingly, in the case that, as shown in FIG. 7, there are twocandidate paths corresponding to a certain path in the first phase andthe branch points differ only in the last level, given the path L1 isthe best matching, and the path L2 is the second matching. The bestmatching refers to the candidate path with the highest similarities andthe second matching refers to the candidate path with the second highestsimilarities.

The calculation unit 13 of the embodiment may calculate, per candidatepath, the similarities between the paths of the tree structures based ona set of feature quantities of the various branch points constitutingthe paths and the determination unit 14 may perform position matching onthe paths based on the similarities per candidate path, thus theposition matching between the images containing the tree structures canbe performed fully automatically and highly precisely.

Third Embodiment

For the above-described first and second embodiments, the case that theacquisition unit 11 extracts the tree structures from the photographicimages with the conventional method of extracting tree structures hasbeen described above; however, the acquisition unit 11 may extract treestructures not with the conventional method of extracting a treestructure but with a new method. Such an embodiment in which a treestructure is extracted with the new method will be described as a thirdembodiment. The operation of the acquisition unit 11 according to thethird embodiment is the operation modified from the operation ofextracting tree structures performed by the acquisition unit 11according to the first or second embodiment. Furthermore, the contentsof the descriptions of the first and second embodiments will not berepeated and the corresponding descriptions in the first and secondembodiments will be cited. In other words, the process of extracting atree structure from a photographic image performed by the acquisitionunit 11 will be described below and descriptions of other processes willbe omitted.

FIG. 8 is a block diagram showing exemplary configuration of theacquisition unit 11 according to the third embodiment. The acquisitionunit 11 of the embodiment also includes a multi-level acquisition unit15 that divides a tree structure into a plurality of levels towardoutside from the root of the tree structure, taking the level where theseed points locate as start points, acquires the corresponding portionsof the tree structure in the respective levels in turn, and combines theacquired portions into a complete tree structure; and a methoddetermination unit 16 that evaluates the acquisition result on theformer level and determines the acquisition method for the current levelbased on the result of the evaluation. The multi-level acquisition unit15 and the method determination unit 16 are realized by the processor.

Moreover, the method determination unit 16 of the embodiment employs anarea growth method as the acquisition method for the level where theseed points. The method determination unit 16 repeats the process ofadaptively acquiring the result of the current level by adaptivelyadjusting the parameters of the current level based on the feedback ofthe segmentation results on the former level until the growth of thewhole tree structure has been completed.

In the following, a specific example of extracting multi-level treestructures by the acquisition unit 11 according to the third embodimentwill be described with reference to FIG. 9.

FIG. 9 is a flow chart of exemplary flow of acquiring tree structures onrespective levels according to the third embodiment. As shown in FIG. 9,first, the multi-level acquisition unit 15 inputs 3D lung CT data (stepS101) and finds seed points in the trachea (step S201). Taking the seedpoints as the start points, the multi-level acquisition unit 15 thenobtains the segmentation results on the first level (step S301). Themulti-level acquisition unit 15 extracts all the large bronchi as a setof seed points (step S401), then further extracts surrounding smallbronchi (step S501), and completes an airway tree (step S601).

In a specific example, the method determination unit 16 obtains thesegmentation results (mainly including the levels 0 to 1 of the tracheatree structure) on the first level from the seed points with a 2D/3Dcombined region growing, estimates the range of the volume for theoverall tree structure according to the segmentation results on thefirst level, adaptively adjusts the parameters used in the growingprocess based on the range of the volume, thereby acquiring the mainfragments (levels 1-4) of the whole tree structure. On the basis ofsegmentation results on the first level and the second level, thecombination of parameters reconstructed morphologically is adjusted tofurther obtain the segmentation results on small bronchi.

The parameters, such as the range of the volume of the whole treestructure, the range of CT value, CT mean value, are used when thefeedback is from the first level to the second level. The parameters,such as the volume of the initial tree structure and the CT mean value,are used when the feedback is from the second level to the third level.

At step S301 described above, the multi-level acquisition unit 15obtains the segmentation results (e.g., the trachea and part of theprimary bronchi) on the first level (0 level or 0-1 level). In otherwords, in step S301, the multi-level acquisition unit 15 extracts thetree structure of the first level. Specific examples of the process insteps S301, S401, and S501 will be described below.

For example, in step S301, the method determination unit 16 calculates amean value M of the CT values of the pixels in a certain area around aseed point (hereinafter, referred to as a CT mean value) in atwo-dimensional image containing the seed point. The methoddetermination unit 16 then takes a point having the CT mean value as anew seed point from the above-described certain area. Taking the seedpoint determined by the method determination unit 16 as a start point inthe lung three-dimensional lung CT data, the multi-level acquisitionunit 15 then takes the area of pixels having CT values within the rangefrom (M−α) to (M+α) (hereinafter, referred to as the CT-value range) asthe tree structure of the first level.

At step S401, the multi-level acquisition unit 15 takes the treestructure of the first level that is extracted at step S301 as a set ofseed points and, taking each of the seed points as a start point,extracts the tree structure of the whole large bronchi of the secondlevel by region growing. Taking the tree structure of the large bronchiin a level equal to or less than the second level extracted at step S401as a set of seed points, the multi-level acquisition unit 15 extractsthe small bronchi around the large bronchi by region growing (stepS501), thereby completing the tree structure (step S601).

In a specific example, the multi-level acquisition unit 15 obtains thesegmentation results (mainly including the level 0 or 0-1 of treestructure) on the first level from the determined new seed point byregion growing where two-dimensional and three-dimensional images are tobe processed as described above. Based on the segmentation results onthe first level, the method determination unit 16 estimates the range ofvolume of the whole tree structure. Based on the estimated volume range,the above-described CT-value range, and the above-described CT meanvalue, the method determination unit 16 then adaptively adjusts theparameters used in the growing process for the second level followingthe first level. For example, the method determination unit 16 uses theabove-described volume range, CT-value range, and CT mean value asparameters used in the growing process for the second level. Themulti-level acquisition unit 15 takes the tree structure of the firstlevel as a set of seed points and, taking each of the seed points as astart point, grows the 3D gray area with the above-described volumerange, CT-value range, and CT mean value as the parameters used in thegrowing process for the second level, thereby obtaining the mainfragments (large bronchi: levels 1 to 4) as the segmentation results onthe second level.

Based on the results of segmentation on the first level and the resultsof segmentation on the second level, the method determination unit 16adaptively adjusts the parameters used in the growing process for thethird level following the second level. For example, the methoddetermination unit 16 uses the volume of the initial tree structure andthe CT mean value as parameters used in the growing process for thethird level. The volume of the initial tree structure refers to, forexample, the volume of the whole tree structure that is estimated basedon the segmentation results on the first level. The multi-levelacquisition unit 15 takes the tree structure of the second level as aset of seed points and, taking each seed point as a start point andusing the volume of the above-described initial tree structure and CTmean value as the parameters used in the growing process for the thirdlevel, acquires the tree structure (small bronchi) of the third level asthe segmentation results on the third level.

That is, the abundant levels of the tree structures of trachea and thehighly accurate results of segmentation of small bronchi are secured byresponding to feedback in different levels.

The tree structure extraction method according to the embodiment has thefollowing effect. The maximum value and the minimum value may be definedprecisely from the feedback of the first level as the tree structuregrowing process can be controlled effectively by adjusting theparameters adaptively. Further, because the tree structure has alreadyincluded most of the bronchi of the (sub-) segment level, themorphological method of extracting a tree structure by a small number ofoperations according to the embodiment sufficiently saves a lot of time.

In short, the medical image processing apparatus 1 according to theembodiment automatically extracts the airway tree structure from thethree-dimensional CT images by extracting the airway tree structures ofa plurality of levels by internal feedback. Accordingly, the results onmultiple levels and with a low error rate may be obtained. Furthermore,the medical image processing apparatus 1 has robustness to differentdiseases and/or all kinds of scanning conditions.

Moreover, the tree structure extraction method of the embodiment usesthe results on the whole first level to grow the tree structure, whichleads to high robustness. While one seed point is necessary to start thealgorithm, but the result does not depend on the selection of the seedpoint. For example, as described above, according to the embodiment, themean value M of CT values of the pixels in the certain area around thefirst seed point is calculated and a point having the CT mean value istaken as a new seed point from among the certain area. In the case thatthe CT value of the pixel of the first seed point is extremely higher orlower than those of the surrounding pixels and the first seed point isdirectly used to perform region growing, tree structure of bronchi ofthe first level is not necessarily extracted accurately; however,according to the embodiment, as described above, a point having the CTmean value is taken as a new seed point and the determined seed point isused, which enables accurate extraction of the tree structure ofbronchi.

Fourth Embodiment

For the first to third embodiments, the case that position matching isperformed over the whole lung (bronchi) has been exemplified.Alternatively, position matching may be performed per area, such as lunglobe. Such an embodiment will be described as a fourth embodiment. Themedical image processing apparatus 1 according to the fourth embodimentfurther includes a division unit in addition to each unit according tothe first, second, and third embodiments and thus is capable ofperforming position matching on a certain selected area.

FIG. 10 is a block diagram showing an exemplary configuration of themedical image processing apparatus according to the fourth embodiment.As shown in the example in FIG. 10, the medical image processingapparatus 1 according to the fourth embodiment further includes adivision unit 17 that divides the tree structure into a plurality ofareas prior to the process by the extraction unit 12, so the extractionunit 12 may extract the branch points for each area according to thevarious areas divided by the division unit 17. The division unit 17 isrealized by the processor.

Here, the division unit 17 can divide the tree structure into aplurality of areas according to the lung lobe, which is based on thepartition of the anatomy, so that position matching can be performed inthe subsequent process according to each of the areas of lung lobe. Suchposition matching in lung lobe level is referred to as lung lobeposition matching. The processing amount of lung lobe position matchingis less than that of position matching over the lung.

Moreover, the division unit 17 can divide only the area of concernaccording to the user's indication or results from other analysisapparatus and so on, thereby the extraction unit 12 extracts branchpoints only in the area of interest.

FIG. 11 is a diagram for explaining exemplary position matchingperformed in a certain area by the medical image processing apparatus 1.As shown in FIG. 11, the division unit 17 divides the lung lobe. In thefollowing process, as described above, the extraction unit 12 extractsbranch points from the divided lung lobe and, as described above, thecalculation unit 13 calculates the similarities between branch points ina first photographic image and branch points in a second photographicimage based on the amounts of characteristics of the branch points. Thedetermination unit 14 then determines the corresponding path between thebranch points in the first photographic image and the branch points inthe second photographic image based on the similarities between thebranch points as described above.

According to the embodiment, position matching is performed not on thewhole tree structure but in a local area. For this reason, with theresults of position matching, the same marker (i.e., biological featurepoint) can be found more accurately in the images in different phases.

Modification

Various embodiments are described above, but the various embodiments arenot limited to the above-described configuration, and they may bemodified as follows.

For example, in the above-described embodiment, the determination unit14 may also determine the branch points extracted by the extraction unit12 as the branch point candidates in the determination step S4 and rankthe various branch point candidates based on the similarities betweenthe branch points in the first photographic image and the branch pointsin the second photographic image.

Further, for example, in the above-described embodiment, the calculationunit 13 is capable of calculating the similarities between variousbranch points after weighting the various branch points according to theposition of the branch points in the tree structure.

Further, for example, in the above-described embodiment, the calculationunit 13 is capable of calculating only the similarities between thebranch points in the same level after dividing the tree structure intodifferent levels according to the level of the tree structure.

The various embodiments of the present invention are described above.The various embodiments acquire the photographic images containing treestructures of an examined subject in different phases, extract theamount of feature of the branch points in the central line of the lung(or lung lobe) by searching for the tree structure according to the treestructure of bronchi, performs position matching by using theinformation, such as the coordinates of branch points, length and angleof segment, similarities between a certain bronchial cross section and abronchial cross section containing branch points, cross-sectional areaof the bronchus, bronchial cross-sectional shape, thereby enabling fullyautomatic and highly precise position matching between the 3D imagescontaining the tree structures.

At least one of the medical image processing apparatus and the medicalimage processing method enables a reduction in the amount of calculationas well as securing of accuracy of image matching.

Some embodiments of the invention have been described, but theembodiments are only for the purpose of exampling, and are not intendedto limit the scope of the invention. These embodiments can beimplemented by a variety of other modes. Further, various omissions,modifications and alterations can be performed without departing fromthe scope of the subject of the invention. Such embodiments and theirmodifications are included in the scope and subject of the invention,and are also included in the scope of the appended claims or theequivalents thereof.

What is claimed is:
 1. A medical image processing apparatus, comprising:acquisition circuitry configured to acquire a first photographic imageand a second photographic image that contain tree structures of asubject; extraction circuitry configured to extract branch points ofeach of the tree structures in the first photographic image and thesecond photographic image; calculation circuitry configured to calculatethe similarities between the branch points in the first photographicimage and the branch points in the second photographic image based onthe feature quantities of the branch points; and determination circuitryconfigured to determine the corresponding path between the branch pointsin the second photographic image and the branch points in the firstphotographic image based on the similarities between the branch pointscalculated by the calculation circuitry.
 2. The medical image processingapparatus according to claim 1, wherein the determination circuitry isconfigured to determine the corresponding branch points between thefirst photographic image and the second photographic image based on thesimilarities between the branch points calculated by the calculationcircuit, and take the path consisting of the various determined branchpoints as the corresponding path between the tree structures.
 3. Themedical image processing apparatus according to claim 1, wherein whenthe determination circuitry is configured to determine a plurality ofpaths, the calculation circuitry is configured to calculate thesimilarities between a reference path in the first photographic imageand each of the paths in the second photographic image, based on a setof the feature quantities of various branch points constituting thepaths, and the determination circuitry is configured to determine thecorresponding path between the tree structures, based on thesimilarities between the various paths calculated by the calculationcircuit.
 4. The medical image processing apparatus according to claim 1,wherein the acquisition circuitry is configured to include: multi-levelacquisition circuitry configured to divide a tree structure into aplurality of levels, taking the level where the seed points locate asstart points, acquire the corresponding portions of the tree structurein the respective levels in turn, and combine the acquired portions intoa complete tree structure; and method determination circuitry configuredto evaluate the acquisition results on the former level and, based onthe result of the evaluation, determine the acquisition method to beperformed by the multi-level acquisition circuitry for the currentlevel.
 5. The medical image processing apparatus according to claim 4,wherein the method determination circuitry is configured to take theacquisition method for the level where the seed points locate as regiongrowing and repeat a process of adaptively acquiring the segmentationresults of the current level by adaptively adjusting the parameters ofthe current level based on the feedback of the segmentation results onthe former level until the growth of the whole tree structure has beencompleted.
 6. The medical image processing apparatus according to claim1, further comprising division circuitry configured to divide the treestructure into a plurality of areas, wherein the extraction circuitry isconfigured to extract the branch points in the areas based on the areasdivided by the division circuitry.
 7. The medical image processingapparatus according to claim 1, wherein the division circuitry isconfigured to determine an area of concern, and the extraction circuitis configured to extract the branch points for only the area of concern.8. The medical image processing apparatus according to claim 1, whereinthe determination circuitry is configured to take the branch pointsextracted by the extraction circuit as branch point candidates and rankvarious branch point candidates based on the similarities between thebranch points.
 9. The medical image processing apparatus according toclaim 1, wherein the calculation circuitry is configured to calculatethe similarities between the various branch points based on thepositions of the branch points in the tree structure after weighting thevarious branch points.
 10. The medical image processing apparatusaccording to claim 1, wherein the calculation circuitry is configured todivide the tree structure into levels based on the levels of the treestructure and calculate only the similarities between the branch pointsin the same level.
 11. A medical image processing apparatus comprising:multi-level acquisition circuitry configured to divide a tree structureinto a plurality of levels, taking the level where the seed pointslocate as start points, acquire the corresponding portions of the treestructure in the respective levels in turn, and combine the acquiredportions into a complete tree structure; and method determinationcircuitry configured to evaluate the acquisition results on the formerlevel and, based on the results of the evaluation, determine theacquisition method to be performed by the multi-level acquisitioncircuitry for the current level.
 12. A medical image processing method,comprising: by acquisition circuitry, acquiring a first photographicimage and a second photographic image that contain tree structures of asubject; by extraction circuitry, extracting branch points of each ofthe tree structures in the first photographic image and the secondphotographic image; by calculation circuitry, calculating thesimilarities between the branch points in the first photographic imageand the branch points in the second photographic image based on thefeature quantities of the branch points; and by determination circuitry,determining the corresponding path between the branch points in thefirst photographic image and the branch points in the secondphotographic image based on the similarities between the branch pointscalculated by the calculation circuitry.
 13. A medical image processingmethod, comprising: by multi-level acquisition circuitry, dividing atree structure into a plurality of levels, taking the level where theseed points locate as start points, acquiring the corresponding portionsof the tree structure in the respective levels in turn, and combiningthe acquired portions into a complete tree structure; and by methoddetermination circuitry, evaluating the acquisition results on theformer level and, based on the results of the evaluation, determiningthe acquisition method to be performed by the multi-level acquisitioncircuitry for the current level.